WORLD INTELLECTUAL PROPERTY ORGANIZATION 
International Bureau 




PCT 

INTERNATIONAL APPLICATION PUBLISHED UNDER THE PATENT COOPERATION TREATY (PCT) 



(51) International Patent Classification 6 : 
G10L 3/00 



Al 



(11) International Publication Number: 
(43) International Publication Date: 



WO 99/35637 

15 July 1999 (15.07.99) 



(21) International Application Number: PCT/US98/27648 

(22) International Filing Date: 30 December 1998 (30.12.98) 



(30) Priority Data: 
09/002,436 
09/133,724 



2 January 1 998 (02.0 1 .98) US 
13 August 1998 (13.08.98) US 



(71)(72) Applicants and Inventors: MATULICH, Richard 
[US/US]; 13409 Midland, no. 134, Poway, CA 92064 
(US). LIGI, Allan [US/US]; 14356 Crcstwood Avenue, 
Poway, CA 92064 (US). 

(74) Agents: MARTIN, Neil, F. et al.; Brown, Martin, Haller & 
McClain, 1660 Union Street, San Diego, CA 92101-2926 
(US). 



(81) Designated States: AL, AM, AT, AU, AZ, BA, BB, BG, BR, 
BY, CA, CH, CN, CU, CZ, DE, DK, EE, ES, FI, GB, GD, 
GE, GH, GM, HR, HU, ID, IL, IN, IS, JP, KE, KG, KP, 
KR, KZ, LC, LK, LR. LS, LT, LU, LV, MD, MG, MK, 
MN, MW, MX, NO, NZ, PL, PT, RO, RU, SD, SE, SG, 
SI, SK, SU TJ, TM, TR, TT, UA, UG, US. UZ, VN. YU, 
ZW, ARIPO patent (GH ( GM, KE, LS, MW, SD, SZ, UG, 
ZW), Eurasian patent (AM, AZ, BY, KG, KZ, MD, RU, TJ, 
TM), European patent (AT, BE, CH, CY, DE. DK, ES, FI, 
FR, GB, GR, IE, IT, LU, MC, NL, PT, SE), OAPI patent 
(BF, BJ, CF, CG. CI, CM, GA, GN, GW, ML, MR, NE, 
SN, TD, TG), 



Published 

With international search report. 



(54) Title: VOICE ACTIVATED SWITCH METHOD AND APPARATUS 




ANALOG 
DC POWER 
SUPPLY 



DIGITAL 
OC POWER 
SUPPLY 



26' 





AC 




DETECT 


.0^ 





32 



AC TO DC 
POWER 
SUPPLY 





AC 




SOURCE 



-24 



(57) Abstract 

A voice activated device for producing control signals in response to speech is self-contained and requires no additional software 
or hardware. The device may be incorporated into a housing that replaces a wall switch that is connected to an AC circuit. An alternate 
housing is portable and includes a jack that plugs into and lies flush against a standard AC utility outlet, and at least one plug for accepting 
an AC jack of any electronic product or appliance. The device acts as a control interface between utility power and connected electrical 
devices by connecting or disconnecting power to the electrical devices based on speech commands. 
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VOICE ACTIVATED SWITCH 
METHOD AND APPARATUS 

This is a continuation-in-part of co-pending application Serial No. 
5 09/002,436, filed January 2, 1998. 

Field of the Invention 

This invention relates generally to voice activated devices for 
producing control signals, and more specifically to a voice activated 
10 switch for producing control signals to switch on or switch off AC 
electrical devices. 

Background of the Invention 

The use of speech recognition technology is becoming a viable 

1 5 means to control one's environment. As the sophistication of speech- 
activated technology increases and the cost of the associated hardware 
and software decreases, the use of speech-controlled devices will be 
commonplace. Applications for speech recognition technology are 
numerous and include the control of appliances, consumer electronics, 

20 toys, and tools. Products and services employing speech recognition are 
developing rapidly and are continuously applied to new markets. 

The use of speech recognition is ideal wherever the hands and/or 
the eyes are busy. Speech commands are a quick, hands-free way to 
control electrical devices. The dangers associated with walking into a 

25 dark room, or the inconveniences of interrupting tasks in order to turn on 
appliances or lights, are alleviated by the utilization of speech recognition 
technology. 

Speech recognition technology has been in development for more 
than 25 years resulting in a variety of hardware and software tools for 
30 personal computers. In a typical application, a speech recognition circuit 
board and compatible software programs are inserted into a computer. 
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These add on programs, which operate continuously in the background of 
the computer's operating system, are designed to accept spoken words 
and either execute the spoken command or convert the words into text. 
The disadvantage in using this approach to control individual appliances is 
5 the necessity of one or more computers. Also, it is unlikely that 

manufacturers will add full blown computer systems to control appliances 
such as washing machines or electronic products such as stereos. 
Computer controlled systems that utilize speech recognition have been 
employed to control the appliances and electronics throughout a house or 

10 building, however, these systems are expensive, complicated, and require 
custom installation. 

Remotely controlling an electrical appliance is currently possible 
using devices employing a variety of technologies. Products using 
acoustic signals are available on the market to control electrical 

15 appliances. These devices recognize specific sounds such as claps, and 
respond by toggling power switches. One drawback of utilizing an 
acoustic device is that it does not provide "hands-free" control. Also, the 
user must remember an acoustic code, such as a sequence of claps, for 
each appliance. 

20 Another way to control an appliance is by the utilization of a remote 

control. Remote control units utilizing speech recognition have been 
designed for electronic products such as VCRs. The speaker talks into a 
control unit while depressing a switch, and the speech commands are 
recognized and transmitted to the VCR using infra-red signals. Although 

25 this system offers a means for the remote control of electronics, it does 
not offer a hands-free solution. Additionally, the user must have the 
remote control unit with him or her, and each target appliance must be 
adapted to receive IR signals. 

Similar to any developing technology, speech recognition poses 

30 many hurdles, including designing the most effective user interface, and 
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increasing response accuracy. A non-friendly user interface is likely to 
frustrate the user when non-responsiveness of the device is the only 
indication of a recognition error. Another difficulty involves 
extemporaneous conversations and sounds that may falsely trigger a 
device response. Speech recognition devices have attempted to 
overcome this problem by allowing a very limited number of speech 
commands such as "ON" and "OFF." However, these devices must be 
programmed with the voices of the speakers that will use the device, and 
do not anticipate noisy environments in which the device is required to 
distinguish between the speaker and other noises. Also, the limited 
vocabulary allows the utilization of one device per room, unless the 
speaker desires to turn on all appliances at the same time. 

The current technology for the remote control of electronic 
consumer products fails to provide a hands-free, economical, compact, 
and easy-to-use device. Additionally, available designs do not offer 
solutions for inaccuracy due to false response, user frustration, and 
ambient noise interference. These problems and deficiencies are clearly 
felt in the art and are solved by the present invention in the manner 
described below. 

SUMMARY OF THE INVENTION 

It is an advantage of the present invention to provide a compact, 
stand alone, speech recognition circuit to control a variety of electrical 
devices, including consumer electronic products or appliances without the 
need for a host computer. 

It is another advantage to provide an easy-to-use device that is 
programmable to recognize a variety of command words so that more that 
one device can be utilized within one room. 
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It is yet another advantage of the present invention to provide a 
low-cost replacement for a standard wall switch and switch box for 
speech control of electrical devices connected to the wall switch circuit. 

It is still another advantage to provide a portable speech recognition 
5 interface between a standard AC outlet and an electrical device. 

A further advantage of the invention is to provide a speech 
recognition device that incorporates user interfaces for confirming 
acceptance of speech commands thereby increasing recognition accuracy 
while reducing the necessity for training the user. 

10 In a preferred embodiment of the present invention, a stand-alone, 

programmable speech recognition device acts as a control interface 
between a 120 V or 230 V AC switch and a connected electrical 
appliance or light. In a preferred embodiment ("wall switch 
embodiment"), the voice activated control circuitry is designed to fit into a 

1 5 switch box shell that can be installed in place of a standard wall switch. 
In an alternate preferred embodiment ("outlet embodiment"), the voice 
activated control circuitry is encased in a portable, palm-sized shell that 
can be plugged into a standard outlet. 

In a wall switch embodiment, the speech recognition circuitry of the 

20 invention is contained on a circuit board having dimensions to fit within a 
standard wall switch box. The circuit board has connections for user 
interfaces including input leads for a microphone for accepting a voice 
command, a manual switch controller for accepting manual operation of 
the switch, and at least one light-emitting diode ("LED"). 

25 The manual switch controller provides a manual means for 

operating the switch, and operates in cooperation with the speech 
recognition circuitry. A variety of technologies can be utilized for the 
manual switch including rocker switches, actuator-type controls, push- 
buttons and touch plate technology. The preferred embodiment utilizes 

30 capacitance touch plate technology that is known in the art. 



WO 99/35637 



PCT/US98/27648 



-5- 

The speech recognition device operates in a continuous listening 
mode which allows it to actively listen for sounds at all times. Ideally, the 
device is located in a position that is exposed and not hidden behind an 
object such as a piece of furniture. An exposed location allows a built-in 
5 microphone to pick up un-muffled sounds and speech in proximity to the 
device thereby increasing the response accuracy. The preferred wall 
switch embodiment is typically placed in a convenient location within a 
room and positioned at approximately four feet (122 cm) from a floor. 
Thus, the microphone will be at an optimal level to accept a speaker's 

10 commands, particularly in circumstances in which the speaker is seated. 
Obviously, where an AC outlet or a light is controlled by more than one 
wall switch, a microphone of at least one of the voice activated wall 
switches is more likely to be in proximity to the user (speaker). 

The outlet embodiment of the present invention is plugged into an 

1 5 AC outlet. The outlet embodiment has at least one plug for accepting the 
cord jack of an electrical device and may necessarily be plugged into an 
outlet behind an object that obstructs the line from the user to the device. 
Therefore, this embodiment may include a separate attachable microphone 
that is placed in a location most likely to maintain an unobstructed line 

20 between the microphone and the user. The use of a separate microphone 
allows the microphone to be placed in a convenient location that is in 
close proximity to the user. This is particularly useful where the 
environment is noisy, or where the user is disabled or has low mobility. 
In other embodiments of the outlet and wall switch devices, the 

25 microphone circuit includes a receiver for receiving transmitted radio 
frequency signals from a separate remote microphone. These 
embodiments are desirable for users who cannot effectively trigger the 
speech recognition because they are not in proximity to the device. For 
example, a user who is seated in a position outside the range of the 



WO 99/35637 



PCT/US98/27648 



-6- 

microphone will be unable to control the device. An RF receiver will 
provide remote speech control of the speech recognition device. 

The voice-activated device is continuously listening for an 
acceptable speech command as long as power from the utility main is 
available. Thus the device is constantly processing background noises 
and establishing an ambient noise level. The ambient noise level is an 
average decibel level of the sounds in the frequency range of speech that 
are detected by the device. For example, a background noise level of a 
50 decibel air conditioning unit causes the device to establish an ambient 
noise level of 50 decibels. Detected sounds below that level are ignored, 
and in order for the device to act upon a command word, the user must 
speak above that decibel level. Establishing an ambient noise level 
enables the device to be used in noisy environments. 

Upon receiving a signal in the frequency range of speech that is 
louder than the ambient level, the device determines whether the signal is 
a valid command word. A valid command word is a member of either a 
set of pre-programmed speaker independent words, or a set of user 
programmed speaker dependent words. These sets of command words 
correspond to two modes of operation known in the art as "speaker 
independent" and "speaker dependent" operation. The user has a choice 
of the mode of operation upon resetting the device. In the preferred wall 
switch embodiment of the invention, reset is activated by pressing the 
touch plate a specified number of times. Reset of the outlet embodiment 
occurs when the device is initially plugged into an outlet. 

The first mode of operation is a speaker independent mode. In this 
mode the device can be used by various speakers and does not have to be 
trained to recognize individual voices. Therefore, the device is pre- 
programmed to respond to a large variety of speech patterns, inflections, 
and enunciations of the target command word. This mode of operation 
usually has a lower number of valid command words than a speaker 
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dependent systems that require more memory to store the various speech 
patterns. In the preferred embodiments, speaker independent command 
words include a name of an electrical device such as "LIGHTS" followed 
by action command words such as "ON" or "OFF" or "DIM." 
5 A speaker dependent mode of operation recognizes only one 

speaker, or a limited number of speakers at a time. The speaker 
dependent mode is activated by resetting and "programming" the device. 
After detecting a reset condition, the device listens for a request to select 
the speaker dependent mode, and the user follows instructions to program 

10 the command words. In a preferred embodiment the user is prompted by 
the device through use of a user interface which includes prompts from an 
indicator such as an LED, or speech instructions from the device itself, or 
both. The device, operating in a speaker dependent mode of operation, 
achieves a high accuracy of word recognition. The disadvantage to using 

1 5 this mode of operation is that the system response accuracy is limited to 
the user who programmed the valid command set. 

The device limits user frustration by signaling an acceptance of a 
valid command word through a user interface that includes an indicator 
such as an LED, or a speaker for communicating speech prompts. The 

20 feedback of the user interface permits the user to adjust his or her 

command word enunciations and inflections which results in a higher 
response accuracy. Once the device recognizes and indicates acceptance 
of a valid command word, the user responds with an action command 
word such as "ON." If the action command word is within the set of valid 

25 command words, the device will respond by performing the desired 

action. For example, in the preferred wall switch embodiment, the device 
responds to the action command word by connecting power or 
disconnecting power to an electrical circuit that is connected to the wall 
switch. For applications where the action command word is meant to dim 

30 ar brighten lights, the device responds by connecting AC at a reduced or 
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increased voltage. In an alternate mode of operation, the action command 
word in not used, and the command word such as "LIGHTS" is repeated to 
toggle the lights on or off. 

In a another embodiment of the present invention, the device 
incorporates current carrier modulation techniques as disclosed in U.S. 
Patent 3,818,481 of Dorfman, which patent is incorporated herein by this 
reference. Using this technology, the device recognizes a variety of 
electrical product command words, where only one command word is 
valid for the attached product. Other valid command words are 
transmitted over the utility main to a second device directly connected to 
the utility main or plugged into an AC outlet. The second device 
demodulates the command word and makes a determination of whether 
the command word is contained within its set of valid command words, 
and whether the command word corresponds to its attached product. 

The present invention provides a compact, continuously listening 
speech recognition circuit that may be incorporated into a variety of 
designs including wall switches and portable outlet devices. A voice 
activated wall switch or wall outlet provides an improved method for 
controlling electrical devices. Limitations of the prior art, including the 
need for complex computer-controlled systems, user frustration, use in 
noisy environments, and limited speech command sets, are overcome by 
the present invention to increase response accuracy and device utility. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Understanding of the present invention will be facilitated by 
consideration of the following detailed description of preferred 
embodiments of the present invention taken in conjunction with the 
accompanying drawings, in which like numerals refer to like parts, and in 
which: 

Figure 1 is a block diagram of a speech controlled device; 
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Figure 2 is a perspective drawing of a portable speech activated 
device; 

Figure 3a is a flowchart of a minimal functionality of a programming 
code for a preferred embodiment of the speech control 
device; 

Figure 3b is a continuation of the flowchart of Figure 3a for an 

on/off application; 
Figure 3c is a continuation of the flowchart of Figure 3a for a 

dimmer application; 
Figure 4 is a front view of a wall switch of a preferred embodiment; 
Figure 5 is a side view of a wall switch of a preferred embodiment; 
Figure 6 is a block diagram of a speech activated wall switch; 
Figure 7a is a flowchart of a wall switch of a preferred 

embodiment; 

Figure 7b is a continuation of the flowchart of Figure 7a for a 

default mode of a preferred embodiment; 
Figure 7c is a continuation of the flowchart of Figure 7a for a first 

user independent mode; 
Figure 7d is a continuation of the flowchart of Figure 7a for a 

second user independent mode; 
Figure 7e is a continuation of the flowchart of Figure 7a for a third 

user independent mode; 
Figure 7f is a continuation of the flowchart of Figure 7a for a 

speaker dependent mode; 
Figure 7g is a continuation of the flowchart of Figure 7f ; and 
Figure 7h is a continuation of the flowcharts of Figure 7f and 7g. 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 

"Speech recognition" refers to the ability of a device to recognize 
what words have been spoken and to take specific actions according to 
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those words recognized. Figure 1 is a block diagram of a preferred 
embodiment of the electrical components of a speech recognition device. 
A microphone 2 accepts and converts speech and other sounds into 
electrical audio signals. The electrical audio signals at the output of the 
5 microphone 1 are amplified by an input amplifier 4 and fed into a 

bandpass filter 6. The band pass filter 6 of the preferred embodiment is 
designed to filter signals outside of the frequency range of approximately 
580 Hz to 4.2 kHz, which represents the typical frequency range of 
speech. 

10 The filtered audio signal is introduced into an automatic gain control 

circuit 8. The phrase "automatic gain control" usually refers to a feedback 
loop that accepts a varying input signal and uses feedback to maintain a 
constant output signal. The automatic gain control circuit 8 of the 
preferred embodiment operates in a different manner by supplying a 

1 5 continuous ambient level signal to the microcontroller 20 over a pre- 
determined time window. The microcontroller 20 maintains the ambient 
level during the time window by sending feedback signals 36, 38 to the 
automatic gain control circuit 8. The ambient level is used as a starting 
level for recognizing speech. Any background noise received by the 

20 device during the time window that is below the ambient level is ignored. 
After the time window has expired, the device establishes a new ambient 
level signal. 

Establishing an ambient level is desirable feature in noisy 
environments because the user need only speak above the ambient level 

25 to trigger the device. Additionally, the ambient level increases the device 
accuracy because the device will not falsely trigger in response to a stray 
or constant background noise. In the preferred embodiment, a time 
window has a duration in the range of 5 seconds to 1 minute, where 
approximately 5 seconds is an ideal duration. Obviously, the particular 

30 pre-determined time window may vary with the type of environment, and 
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is not meant to be a limiting factor. For example, in a particularly quiet 
environment, the time window may be of a longer duration because 
changing background noises are not expected. 

In the preferred embodiment, the automatic gain control loop 
includes the automatic gain control circuitry 8, the microcontroller 20, an 
amplifier 10, and a multiplying buffer 12. The output of the automatic 
gain control circuitry 8 is fed into an amplifier 10. The output of the 
amplifier 10 is fed into the microcontroller 20 and a multiplying buffer 12. 
Finally, the output of the multiplying buffer is also fed into the 
microcontroller. Thus, the "ambient level" that is sent to the 
microcontroller 20 consists of both a zero level 40 at the output of the 
amplifier 10, and a multiplied level 42 at the output of the multiplying 
buffer. The microcontroller 20 of the preferred embodiment of Figure 1 is 
a general purpose microcontroller manufactured by Sensory, Inc.™ which 
is configurable for a variety of applications including speech recognition. 

The microcontroller of the preferred embodiment requires the zero 
level 40 and the multiplied level 42 to produce feedback signals 36, 38. 
However, other embodiments of the invention using different 
microcontrollers may have differing input requirements to maintain the 
ambient level that is established using feedback signals 36, 38. Also, the 
automatic gain control circuitry 48 may be deleted from the circuit of 
Figure 1 for embodiments where the microcontroller includes equivalent 
circuitry of the automatic gain control 8, amplifier 10, and multiply buffer 
1 2. For such embodiments, the output of the band pass filter 6 is directly 
accepted by the microcontroller 20. As additional functions become 
available on the microcontroller 20, other circuit functions such as 
program memory band pass filter and input amplifier may eliminate the 
need for separate circuits to provide these functions. 

The microcontroller circuitry includes the microcontroller 20 and a 
number of memory modules. The memory modules of the preferred 
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embodiment include the program memory 22 and speech command 
memory 14 which are shown external to the microcontroller 20, but 
which may be internal to the microcontrollers of other embodiments of the 
invention. The program memory 22 is a Read Only Memory (ROM) 
module which stores the programming code of the microcontroller 20. 
The programming code establishes the sequence of events that are 
followed by the device to produce a control signal 44 in response to valid 
speech commands. The speech command memory 14 of the preferred 
embodiment employs a Random Access Memory (RAM) module which 
stores the speaker dependent speech commands. The speaker 
independent speech commands are stored in a separate memory ROM 
module (not shown) which may be internal to the microcontroller. The 
term "memory module" does not necessarily refer to separate circuit 
elements. For example, all ROM data may be stored in the same circuit 
element, but at different address block locations. 

Power circuitry of the preferred embodiment which supplies analog 
and digital operating voltages to the device circuitry includes an AC 
source circuit 24, an AC to DC power supply circuit 26, an analog DC 
power supply circuit 28, and a digital DC power supply circuit 30. 
Standard utility AC is supplied to the AC source circuit 24 by 
means of a standard AC jack that is plugged in to a standard AC outlet 
72, as shown in Figure 2. The device may be adapted to be compatible 
with a 1 20 V or 230 V AC standard. The AC to DC power supply circuit 
26 convert the standard utility AC to DC voltages which are fed into the 
analog DC power supply circuit 28 and the digital DC power supply circuit 
30. The analog DC power supply circuit 28 supplies power to the input 
amplifier 4 and the microcontroller 20. The Digital DC power supply 
supplies digital voltages to the microcontroller 20. 

Standard utility AC is also supplied to an AC detect circuit 32 
which is connected to the microcontroller 20 and the output control 
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circuit 16. Upon recognition of a valid speech command, the 
microcontroller sends a control signal 44 to the output control circuit 1 6. 
The control signal 44 enables or disables a connection of the standard 
utility AC into the output control circuit 16. In the preferred embodiment, 
the output control circuit 16 includes a power switch that connects the 
standard utility AC to a standard AC plug. 

For applications where the device is used as a dimmer, the AC 
detect circuit 32 synchronizes an AC signal with the standard utility AC to 
produce an AC waveform having an increased or decreased voltage. 
The AC waveform is connected to the output control circuit 1 6 by the 
control signal 44 upon recognition of a valid speech command. 

In a second embodiment of the invention the output control circuit 
16 has modulation capability and can modulate and transmit a control 
signal on the standard utility AC via the AC source circuit 24 using 
current carrier technology as described herein. This capability allows the 
device to remotely control electronics and appliances that are connected 
to the same standard utility AC circuit. The AC detect circuit may also 
include a demodulator to detect and demodulate a signal from the 
standard utility AC. The demodulated signal is sent to the microcontroller 
20 for a determination of whether the demodulated signal represents a 
valid speech command for an electrical appliance that is attached to the 
device. If the signal is a valid speech command, control signal 44 is sent 
to enable the output control circuit 16. 

The user interface circuit 34 of the preferred embodiment is 
connected to the microcontroller 20 and includes an indicating device 
such as an LED, or a speaker, or both. The user interface circuit 34, 
which is provided as a convenience for the user and does not affect the 
operation of the device, informs the user that the device has received 
either an invalid or a valid speech command. Without the visual or audio 
feedback, the user can not be certain of the reason for a non-response of 
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the device. For example, the user may not be enunciating the command 
correctly, or may be using an invalid speech command. Thus, the user 
feedback lessens frustration and leads to an increase in correct device 
responses. 

5 The indicating device includes at least one LED which may be uni- 

or multi-colored to prompt the user, signal an unrecognized command, 
and/or signal acceptance of a valid command. The indicating device may 
also include a display or another means of visually indicating the 
completion of an event. The speaker is connected to the microcontroller 

10 20 via amplifier circuitry known in the art and provides a means for 

communicating spoken instructions or audio prompts which are stored in a 
memory module of the microcontroller 20. Obviously, a combination of 
an LED and a speaker will provide the highest degree of user convenience. 
Figure 2 is a perspective view of one embodiment of a speech 

15 activated device. The device consists of a shell 60 that houses the 

speech recognition circuitry of Figure 1, and is designed to be compact 
and self-contained such that the entire device plugs into a utility outlet 
72. The shell contains several openings for components of a speech 
recognition circuit, and may include openings for an indicating device 64, 

20 a speaker 70, a microphone 62, a microphone plug 68, a standard AC 
utility plug 66, and a standard utility jack (not shown) which plugs into 
the utility outlet 72. The electronic product or appliance cord 74 is 
plugged into a standard AC utility plug 66 which is located on a face of 
the shell 60. 

25 The program stored in the program memory 22 of Figure 1 varies to 

accommodate the available device features and the desired mode of 
operation. Figure 3a is a flowchart of functionality of a sample 
programming code of a preferred embodiment and is not meant to limit 
the possible programming possibilities. Start block 100 represents the 

30 initial power-up of the device after it has been plugged into the utility 
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outlet 72 shown in Figure 2. The start block 100 may also include 
additional routines such as a mode of operation routine that prompts the 
user to record valid speech commands for a speaker dependent 
application. If user response is not forthcoming, the device defaults to a 
speaker independent mode of operation. Once the microcontroller 20 has 
established the operating parameters set forth in the start block 100, the 
microcontroller 20 proceeds to configure its input/output (I/O) ports. The 
I/O configuration is pre-determined and will vary with the parameters 
chosen in the start block 100. 

The indicating device 64 has a default value, and for the minimal 
preferred embodiment, the default is an "off" state in which the indicating 
device 64 is not illuminated. Whether the indicating device 64 is "on" or 
"off" to indicate an active listening state is a matter of preference, and in 
an alternate embodiment, the indicating device 64 is illuminated as the 
default mode to indicate that the device is actively listening. 

Block 1 04, which is the default block for most of the decisions 
blocks of the subsequent programming code, sets the indicating device 
into an "off" state. The first expected command word, which may be one 
of a set of first expected command words, is retrieved in block 106. The 
device waits for a pre-determined silence period of block 108. If there are 
no sounds which are within the frequency of speech and above an 
ambient level for the duration of the silence period, then the silence is 
acceptable 1 10 and the device waits for a first speech utterance 1 1 2. If 
the silence is not acceptable, the program defaults to the default block 
104 and restarts the above process. 

The silence period-is a required limitation of the technology, and as 
the technology improves, the silence period will approach zero. 
Technology that requires the program to pause in-between words is 
referred to as discrete speech or isolated speech technology. Discrete 
speech recognition systems can only recognize words that are spoken 
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separately. In contrast, continuous speech technology does not require 
phrases of natural speech to be broken into distinct words separated by 
silences. The device of the preferred embodiment employs discrete 
speech technology with a silence period on the order of .01 to .07 
5 seconds. This silence period will vary according to the microcontroller 20 
employed. 

Block 1 12 represents the continuously listening feature of the 
device, and the first utterance does not have to occur within a set time 
period. Once an utterance occurs the signal, which is received through 

10 the microphone 2, is recorded in bock 112. If an acceptable recording has 
occurred 1 1 4, then the duration of the word is checked 116. An 
acceptable recording 1 14 is a recording which contains data within the 
frequency range of speech, and a duration 1 1 6 is the actual time that it 
took to utter the word. Typical durations of words are known because 

1 5 the acceptable command words are from a pre-determined set. Thus, the 
utterance can be no longer than the longest valid command word. If the 
utterance is longer than the longest valid command word 1 18, the 
program defaults to default block 104, otherwise, the utterance is 
compared to the words included in the set of valid command words. 

20 The set of valid command words of the minimal preferred 

embodiment includes the word "LIGHTS." If the recorded utterance of 
block 1 1 2 favorably compares with the pre-recorded samples of the word 
"LIGHTS" 122, then the device has found a positive match, and proceeds 
to retrieve the second word values 124 which constitutes a desired action 

25 such as "ON" or "OFF" or "LOW." The silence level is initialized in block 
126, and has a minimum decibel level equal to the ambient noise level 
that is determined over a time window. The device listens for an 
acceptable silence level for a pre-determined time duration 128, and 
illuminates the indicating device. If the silence period is of an acceptable 

30 time duration 132, then the device records the second utterance 1 34. An 
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acceptable recording 136 and utterance duration 138, 140 advances the 
program sequence to a point of recognizing the word 142. If the duration 
of the silence period or the recording is unacceptable, or the duration of 
the word is too long, then the program defaults to block 104, and the 
5 indicating device is turned off and the process starts again. 

Figures 3b and 3c are continuations of the program flowchart in 
Figure 3a, Figure 3b represents the program for a simple "ON" or "OFF" 
application, and Figure 3c describes the program flow for a light dimmer 
application. In the application of Figure 3b, if the second utterance is 

10 "ON" 144, and a switch included in the output control circuit 16 is open 
148, then a control signal 44 is sent to the output control circuit 1 6 to 
close the switch 150. The closed switch connects standard utility AC to 
a utility plug. The program returns to the default block 104, turns off the 
indicating device, and waits for a new sequence of speech commands. If 

1 5 the utterance is not "ON" 144, then the utterance is compared to the pre- 
recorded word "OFF" 152. If the utterance is determined to be a match 
152, then a determination is made regarding whether the switch of the 
output control circuit 16 is closed 154. If the switch is closed 154, a 
control signal 44 is sent to the output control circuit 16 to open the 

20 switch and disconnect AC power from the utility plug 1 56. All other 
outcomes return the program to the default block 104. 

The continuation of the flowchart for a light dimmer application is 
illustrated in Figure 3c. The utterance is compared to pre-recorded words 
including "LOW" 158, "MEDIUM" 160, "OFF" 162, and "ON" 164. If a 

25 match is identified, control signal 44 is sent to the output control circuit 
1 6 to close 1 66, 1 68, 1 72 or open 1 70 the switch. Also, the 
microcontroller 20 communicates with the AC detect circuit 32 to send a 
reduced or increased AC voltage level 166, 168 to the output control 
circuit 16. A non-matching utterance defaults the program to the default 

30 block 104. 
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Figures 1 , 2, 3a, 3b, and 3c illustrate a preferred embodiment of a 
portable, generally palm-sized speech recognition device. The preferred 
embodiment is plugged into a standard wall socket and includes at least 
one AC plug for accepting the cord/jack of any AC operated device. The 
5 speech recognition device is an economical solution to controlling 
electrical devices by speech commands. 

Figures 4 and 5 illustrate another preferred embodiment wherein 
voice activated control circuitry is housed in a wall switch assembly 200 
that includes a switch plate 210 and switch box 218. Figure 4 illustrates 

10 a front view of a wall switch plate 210. The preferred wall switch 
embodiment utilizes a capacitance touch plate 202 as the manual 
switching control. Other embodiments may utilize other touch pad 
technologies or mechanical switches. The switch plate 210 also includes 
a microphone 206 for accepting speech commands. The user interface of 

15 the preferred embodiment utilizes a green LED 208 and a red LED 204 to 
prompt the user and to indicated that the device is actively listening for a 
speech command. Other embodiments of the voice activated wall switch 
may utilize varying user interfaces including one or more LEDs of varying 
colors, one or more multi-colored LEDs, a character display device, a 

20 speaker for audio prompts, or any combination thereof. Standard switch 
plate screws 220 secure the switch plate 210 to the switch box 218. 

Figure 5 is a cross section of the wall switch assembly 200. The 
switch box 218 houses a power circuit board 212 and a speech 
recognition circuit board 216 connected by at least one connector 222. 

25 In the preferred embodiment of the voice activated wall switch, the 

connector 222 includes connections for power signal lines and control 
signal lines. An aluminum base plate 214 provides structural support for 
the components of the switch box assembly 200. In addition, the 
aluminum plate 214 may act as a heat sink for various components on the 
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power circuit board 212 by including wings or tabs that extend from the 
aluminum plate 214 to contact the power components. 

The speech recognition circuit board 216 is a stand-alone item that 
may be incorporated into other electrical or electronic devices including 
5 various wall switch assemblies. Referring to Figure 6, the circuit board 
216 has inputs that connect to a microphone 602 and AC source 624, 
and outputs that connect to one or more user interfaces 634, and a touch 
and dim controller 646 or any other suitable manual switch. Thus, the 
speech recognition circuit board 21 6 may be adapted to a particular 
10 application by connecting the inputs and outputs to appropriate 
components. 

Figure 6 is a block diagram of a preferred embodiment of a speech 
recognition circuit board 21 6 and externally connected components for a 
voice activated wall switch assembly 200 as shown in Figures 4 and 5. . 

15 The purpose and operation of the elements of the block diagram of Figure 
6 are substantially similar to the elements of the block diagram of Figure 
1 . The microphone 602 connects to an input to the speech recognition 
circuit board 21 6 and converts speech and other sounds to electrical 
audio signals. The electrical audio signals are amplified by an input 

20 amplifier 604 and filtered by a band pass filter 606 to exclude frequencies 
outside the frequency range of speech. 

An automatic gain control circuit 648 accepts the filtered audio 
signal from the band pass filter 606 and establishes an ambient noise 
input level for microcontroller 620. In other embodiments, the automatic 

25 gain control circuit 648 may be included in the microcontroller 620 

allowing the output signal from the band pass filter 606 to be directly 
connected to the microcontroller 620. 

The power circuitry of the preferred embodiment resides on the 
power circuit board 212 as shown in Figure 5. The power circuitry 

30 includes an AC source input circuit 624, an AC to DC power supply circuit 
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626, an analog DC power supply circuit 628, a digital DC power supply 
circuit 630, and an AC detect circuit 632. The AC source input circuit 
624 is directly connected to an AC circuit provided to the wall switch. In 
other embodiments, portions of the power circuitry may reside on the 
5 speech recognition circuit board 216. 

The microcontroller circuitry includes the microcontroller 620, 
program memory 622, and speech command memory 614 which are 
shown external to the microcontroller 20, but which may be internal to 
microcontrollers of other embodiments of the invention. The program 

10 memory 622 is a Read Only Memory (ROM) for storing programming code 
of the microcontroller 620. The program memory 622 or an additional 
ROM stores speaker independent words. The speech command memory 
614 of the preferred embodiment stores speaker dependent speech 
commands that are programmed by a user into the device during a 

15 programming mode. 

The user interface 634 of the preferred embodiment of the voice 
activated wall switch assembly 200 includes a green LED 208 and a red 
LED 204 as illustrated in Figure 4. Other embodiments may include a 
single LED, or any other type of indicator that is controllable by the 

20 microcontroller 620.. The user interface 634 provides visual prompts for 
the user to indicate that the circuit is operating and accepting speech 
commands or programming mode inputs of speaker dependent commands. 

Output control signal 644 is generated by the microcontroller 620 
25 for instructing the output control circuitry 616 to switch power on or off, 
to toggle power, or to reduce power, i.e. dim lights. The output control 
circuitry 616 of the preferred embodiment is located on the power circuit 
board 212, and is connected to the speech recognition circuit board 216 
via the connector 222. It should be noted that the division of circuitry 
30 between the speech recognition circuit board 21 6 and the power circuit 
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board 21 2 is a matter of design convenience, only. Other embodiments 
of the voice activated wall switch may vary the locations of the electrical 
components. 

The voice activated wall switch has a manual touch and dim 
controller 646 that includes the touch pad 202 of Figures 4 and 5 for 
manually controlling a switch located in the output control circuitry 616. 
The speech control and the touch control are simultaneously active. In 
addition, the current state of the power is known by the microcontroller 
620. For example, if an electrical device, e.g. a light, is "ON," then 
pressing the touch pad 202 will toggle the power "OFF." If a user 
subsequently uses a speech command "LIGHTS," the power is switched 
"ON." 

Figures 7a through 7h illustrate a flow diagram for the 
microcontroller program ("the program") of a preferred embodiment of the 
voice activated wall switch. Referring to Figure 7a, start blocks 700 and 
702 represent an initial application of AC power to the wall switch 
circuitry that occurs during installation of the device. A red LED 204, as 
shown in Figure 4, is illuminated 704 to indicate that the device has 
power. The green LED 208 is also illuminated 706 to indicate that the 
device is listening for a word 708. If the user does not issue a speech 
command 710, then the program branches to a default mode 712. If the 
user utters a user independent command as shown in decision blocks 
714, 716, 718, then the program branches to the appropriate mode of 
operation. The command "PROGRAM" 720, causes the microcontroller 
620 to initiate a programming mode to learn user dependent commands. 
An unrecognizable command, i.e. a command that is not in microcontroller 
memory, causes the program to branch to default mode 722 shown in 
Figure 7b. 

Figure 7b illustrates a Default Mode 722 of the wall switch of the 
preferred embodiment. The microcontroller 620 determines whether the 
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user has pressed the touch pad three times 724 for the purpose of 
resetting the mode of operation. Other embodiments of the program may 
use a different mode reset requirement, e.g. two quick presses to the 
touch pad rather than three. As described herein, a single press to the 
5 touch pad toggles the state of the applied AC power to "on" or "off." In 
the preferred embodiment of the voice activated wall switch, three 
presses to the touch pad cause the program to branch to the start of the 
mode selection sequence, block 704 of Figure 7a. If a reset condition is 
not detected, the green LED 208 is illuminated 726, and the voice 

10 activated device waits for a command word. If the user says the 
command "LIGHTS" 728, the green LED 208 is turned off and the 
microcontroller 620 sends a control signal 644, as shown in Figure 6, to 
the output control circuit 616, to toggle the power 734. Upon toggling 
the power in block 734, the program loops back to the start of default 

15 mode 722. 

A Lights On/Off Mode is illustrated in Figure 7c. The program 
checks for a reset condition 740. If the user has not initiated a reset, the 
green LED 208 and the red LED 204 illuminate 742 to indicate that the 
voice activated device expects to receive an acceptable first word. Any 

20 detected word other that "LIGHTS" 744, causes the program to branch to 
the start of the Lights On/Off Mode 738. If the word "LIGHTS" is 
detected 744, the green LED 208 is illuminated 748 to indicate that the 
device expects to receive a second acceptable word, "ON" or "OFF" 750. 
If the device does not detect either acceptable word 750, the program 

25 branches to the start of the Lights On/Off Mode 738. If the device 

detects either the word "ON" or "OFF" 750, the green LED 208 is turned 
off and the power is toggled accordingly 754. 

Figure 7d illustrates the user independent Computer Lights On/Off 
Mode 758. Absent a reset condition 760, the device illuminates both 

30 LEDs 204, 208, and waits for an acceptable first word 762. If the word 
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"COMPUTER" is detected 764, the green LED 208 is illuminated 768 to 
indicate that the device is waiting for a second word, "LIGHTS/ 1 "ON," or 
"OFF" 770. Upon detection of an acceptable word, the green LED 208 is 
turned off and the power is toggled accordingly 774. 
5 Figure 7e also illustrates a user independent mode in blocks 788 

through 804. If the user does not reset the mode 790, the LEDs 
illuminate 792. The first expected word is "INTELSWITCH" 794 and the 
second expected word is either "ON" or "OFF" 800. If the words are not 
detected in the appropriate order, the program turns off the illuminated 

10 LEDs 796, 802 and branches to the start of the mode 788. An 

acceptable sequence of commands causes the microcontroller 620 to 
send a control signal 644 to switch the power either "ON" or "OFF" 804. 

The program flowchart of the preferred embodiment of a voice 
activated wall switch illustrates a default mode and three additional user 

1 5 independent modes. The words used as command words for these 
illustrative modes are not meant to be limiting, and other words and 
sequences of words may be programmed into program memory. 

Figures 7f, 7g, and 7h illustrate the flowchart for a speaker 
dependent mode 806. The red and green LEDs 204, 208 are flashed once 

20 to indicate that the device is waiting to record the first word 808. The 
green LED 208 is then illuminated to indicate that the device is listening 
for the first word 810. If the device has not recorded a third silence 
period 812, then the program determines whether an acceptable recording 
has occurred 816, and either branches to the start of the user dependent 

25 mode 808, or indicates a valid recording 818 by flashing the red LED 204. 
Upon detection of a third silence period, the program checks whether 
words have been recorded in memory 814. If memory contains user 
dependent words, the program branches to wait for a user dependent 
command, [f the recorded word is the first recording 820, the program 

30 stores the word pattern in a temporary memory 822 and branches to 808 
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to prompt the user to repeat the word. If the recorded word is the second 
or greater recording 820, the program compares the currently recorded 
word with the previously recorded word 824. If the word patterns match 
826, then the first word is saved in memory 828. If the word patterns do 
5 not match 826, the program branches to 808 to prompt the user to repeat 
the command word until the user can repeat the word in a substantially 
similar manner. 

Figure 7g is a continuation of the user dependent programming 
mode. Once a first word has been successfully recorded, the LEDs are 

10 flashed to indicate that the device is waiting for a second word 834. The 
green LED 208 is illuminated to indicate that the unit is ready to record 
the second word 836. if a silence period is recorded three times 838, 
then the program branches to detect a valid command word. If a valid 
word pattern is recorded 840, the red LED 204 flashes once 842. If the 

1 5 user has not repeated the word 844, the word is recorded in a temporary 
memory 852, and the program branches to the beginning of the sequence 
to record a second word 834, If the user has repeated the word at least 
once 844, the program compares the current word pattern with the 
previously recorded word pattern 846 to determine whether a match 

20 exists 848. If the patterns match 848, the user has successfully 

programmed the second word which is stored in a second position in 
memory 850. 

Once a sequence of user dependent commands is successfully 
recorded, the device is ready to respond to user dependent commands as 
25 illustrated in the flowchart of Figure 7h. If a reset condition is not 

detected 856, the device illuminates the green LED 208 to indicate that it 
is waiting to accept a command 858. If a speech command is 
successfully recorded 860, the LED is turned off 864 and the program 
determines whether the word is a valid user dependent word. If the word 
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is recorded in memory 866, the power is toggled 868. Otherwise, the 
program loops back to 856 to await a valid user dependent command. 

The flowchart of Figure 7h illustrates a minimal example of 
recognizing a single user dependent word. However, the other 
5 embodiments of the program may require a sequence of words before the 
power is toggled 868. The other embodiments of the program may also 
respond according to a particular command such as "DIM" or "ON" or 
"OFF." Thus, the flowchart is presented for illustrative purposes and is 
not meant to limit the breadth of the microcontroller program. 
10 It is evident that there are additional embodiments which are not 

illustrated above but which are clearly within the scope and spirit of the 
present invention. The above description and drawings are therefore 
intended to be exemplary only and the scope of the invention is to be 
limited solely by the appended claims. 

15 



WE CLAIM: 
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CLAIMS 

1 . A device for responding to a speech command of a device 
user, said device comprising: 
5 an AC circuit having means for producing a plurality of 

operating voltages for said circuit, said AC circuit 
having an input of a standard utility AC; 
a microphone circuit having means for producing electrical 
signals in response to sounds; 
10 a filter circuit having means for filtering said electrical 

signals, said filter circuit producing filtered electrical 
signals having frequencies within a frequency range of 
speech; 

a microcontroller circuit having means for detecting a valid 
1 5 speech command of a plurality of valid speech 

commands, said microcontroller having a plurality of 
inputs comprising said filtered electrical signals and a 
plurality of outputs comprising at least one control 
signal for controlling at least one switching means; 
20 said at least one switching means for connecting said 

standard utility AC to said at least one AC circuit; 
a manual control input circuit for manually controlling said at 

least one switching means; and 
at least one indicator connected to said microcontroller, said 
25 at least one indicator for prompting said device user to 

other speech commands when said microcontroller 
circuit is enabled to receive a speed command. 
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2. The device for responding to a speech command as in 
claim 1 f further comprising a shell for encasing said circuit, said shell 
comprising a wall switch assembly. 

5 3. The device for responding to a speech command as in 

claim 1 , further comprising an automatic gain control circuit having means 
for producing an ambient level signal from said filtered electrical signals. 

4. The device for responding to a speech command as in 
10 claim 1, said device further comprising: 

a means for choosing a plurality of modes of operation, a first mode 
of operation of said plurality of modes of operation for 
programming said device with said plurality of valid speech 
commands. 

15 

5. The device for responding to a speech command as in 
claim 1, further comprising a dimmer circuit having means for producing a 
reduced-voltage AC signal, wherein said at least one switching means 
connects said reduced-voltage AC signal to said at least one AC circuit. 

20 

6. The device for responding to a speech command as in 
claim 1 , said device further comprising: 

a means for producing speech instructions; and 

at least one speaker means for outputting said speech instructions. 

25 

7. The device for responding to a speech command as in 
claim 1 , said device further comprising: 

a modulator circuit having means for modulating said 

standard utility AC with a second control signal of said 
30 at least one control signal. 
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8. The device for responding to a speech command as in 
claim 6, said device further comprising: 

a demodulator circuit means connected to said 

microcontroller for demodulating a modulated control 
5 signal from said standard utility AC. 

9. An apparatus for producing a plurality of control signals, 
said apparatus comprising: 

a microphone circuit for receiving sounds and converting said 
10 sounds into electrical signals, said microphone circuit 

having an input amplifier for amplifying said electrical 
signals, and a band pass filter for filtering sounds 
outside a frequency range of speech, said band pass 
filter outputting a filtered signal; 
1 5 a processor circuit comprising: 

a microcontroller having an input of said filtered signal, 
said microcontroller producing at least one AC 
control output signal upon recognition of a valid 
speech command of a set of valid speech 
20 commands, said set of valid speech commands 

comprising user speech commands and pre- 
programmed speech commands; 
a plurality of memory modules for storing data, said 
data comprising: 

25 at least one instruction set for said 

microcontroller; 

said user speech commands; and 

said pre-programmed speech commands; 
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a power circuit for supplying digital and analog power to said 

control circuit, said power circuit receiving an AC input 

from an AC circuit; 
an AC output circuit for connecting an AC signal to at least 
5 one AC circuit, said AC output circuit enabled by said 

at least one AC control output signal; 
a manual control circuit for manually controlling said AC 

output circuit; and 
a casing for holding said control circuit. 

10 

10. The apparatus as in claim 9, wherein said processor 
circuit further comprises an automatic gain control circuit for producing at 
least one ambient level signal from said filtered signal. 

15 11. The apparatus as in claim 9, wherein said casing is 

designed to replace a wall switch assembly. 

12. The apparatus as in claim 9, further comprising at least 
one indicator for prompting a user of said apparatus and for confirming 
20 recognition of said valid speech command. 

1 3. The apparatus as in claim 9, wherein said band pass 
filter filters out sounds outside a frequency range of 580 Hz - 4.2 kHz. 

25 

14. The apparatus as in claim 9, wherein said control circuit 
further comprises a speaker circuit, said speaker circuit for communicating 
speech instructions from said processor circuit. 
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15. The apparatus as in claim 9, wherein said power circuit 
further comprises a dimmer circuit for producing an AC signal having a 
reduced voltage at a standard utility frequency. 

5 16. A method for controlling a device using speech 

recognition, said device having an input of standard utility AC and an AC 
output switch connected to an AC circuit, said method comprising the 
steps of: 

accepting environmental sounds and speech sounds; 
10 converting said environmental sounds and speech sounds 

into a plurality of electrical signals; 
filtering said plurality of electrical signals that are outside a 

range of speech to produce at least one filtered 

electrical signal; 

1 5 comparing said at least one filtered electrical signal with a 

predetermined set of speech sourxis; and 
producing at least one output signal when said at least one 

filtered electrical signal matches said predetermined 

set of speech sounds; 
20 wherein said at least one output signal controls said AC 

output switch. 

1 7. The method for controlling a device as in claim 1 6, 
further comprising the step of: 
25 toggling said AC output switch utilizing a manual control 

means. 

18. The method for controlling a device as in claim 17, 
wherein said step of toggling said AC output switch comprises the step 
30 of: 

pressing a touch pad. 
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1 9. The method for controlling a device as in claim 1 6, 
further comprising the step of: 

illuminating at least one indicator to indicate a ready state for 
accepting said speech sounds. 
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